GENNAPE: Towards Generalized Neural Architecture Performance Estimators
نویسندگان
چکیده
Predicting neural architecture performance is a challenging task and crucial to design search. Existing approaches either rely on predictors which are limited modeling architectures in predefined space involving specific sets of operators connection rules, cannot generalize unseen architectures, or resort Zero-Cost Proxies not always accurate. In this paper, we propose GENNAPE, Generalized Neural Architecture Performance Estimator, pretrained open benchmarks, aims completely through combined innovations network representation, contrastive pretraining, fuzzy clustering-based predictor ensemble. Specifically, GENNAPE represents given as Computation Graph (CG) atomic operations can model an arbitrary architecture. It first learns graph encoder via Contrastive Learning encourage separation by topological features, then trains multiple heads, soft-aggregated according the membership network. Experiments show that NAS-Bench-101 achieve superior transferability 5 different public including NAS-Bench-201, NAS-Bench-301, MobileNet ResNet families under no minimum fine-tuning. We further introduce 3 newly labelled benchmarks: HiAML, Inception Two-Path, concentrate narrow accuracy ranges. Extensive experiments correctly discern high-performance these families. Finally, when paired with search algorithm, find improve while reducing FLOPs three
منابع مشابه
Generalized Empirical Likelihood Estimators
In an effort to improve the small sample properties of generalized method of moments (GMM) estimators, a number of alternative estimators have been suggested. These include empirical likelihood (EL), continuous updating, and exponential tilting estimators. We show that these estimators share a common structure, being members of a class of generalized empirical likelihood (GEL) estimators. We us...
متن کاملGeneralized Maximum Spacing Estimators
The maximum spacing (MSP) method, introduced by Cheng and Amin (1983) and independently by Ranneby (1984), is a general method for estimating param eters in univariate continuous distributions and is known to give consistent and asymptotically efficient estimates under general conditions. This method can be derived from an approximation based on simple spacings of the Kullback-Leibler informat...
متن کاملMultivariate generalized S-estimators
In this paper we introduce generalized S-estimators for the multivariate regression model. This class of estimators combines high robustness and high efficiency. They are defined by minimizing the determinant of a robust estimator of the scatter matrix of differences of residuals. In the special case of a multivariate location model, the generalized S-estimator has the important independence pr...
متن کاملTowards a High Performance Extensible Grid Architecture
Grid computing systems are being positioned as the high performance computing infrastructure of the future that will be used to solve grand challenge problems and also provide the infrastructure for wide area distributed network computing. The architecture of the Grid will determine if it will meet these challenges. We propose a Grid architecture that is secure, high performance and extensible....
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2023
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v37i8.26102